The use of computers to change, augment, and reshape original film and video images has been a practice that has steadily been on the rise. A digital film workstation (DFW) may be used to fix problems that originate during the production stage and to create special effects. Many special effects sequences are achieved by layering different images together in one composite frame. A first film or video clip may show an actor in a first location and a background image of a second location may be layered under the actor. The resulting affect is that the actor appears in the second location. A composite image may be a combination of tens of different image layers depending upon the desired special effect.
Frequently it is desirable to add or remove a portion of an image from one of the layers of the composite image. For example it is often necessary to remove unwanted elements, such as the safety wires securing the stunt people, from film sequences. Adding or removing elements from an image requires that a film editor be able to manipulate individual pixels within the composite image. One software tool that enables a film editor to manipulate individual pixels in the composite image is a layer based paint system.
Typical layer based paint systems include an input device, such as a mouse or pressure sensitive tablet coupled to a pointer on a display screen. The movement of the input device mirrors the movement of the pointer so that the user is given the impression of painting directly on an image on the display screen. The layer based paint system allows users to paint new layers, creating new images with their own image and transparency (alpha) information. The layer based paint system also allows the user to paint on existing layers, changing the image and/or alpha information in the layer to add or remove objects from the layers. Using the layer based paint system, it is also often possible to move the layers relative to each other to give them an offset in X and Y directions.
In the simple layer paint systems, each layer is itself an image. The layers are stored as image rasters with image and transparency (alpha) information. If a process is performed on the layer, the process directly modifies the pixels associated with the layer. When the process has been performed on that layer, the layer is indelibly modified.
One advantage of the simple layer based system is that, since each layer is simply an image raster, it is possible to paint directly onto any layer to change the contents of rasters in any layer. There is always a simple one to one relationship between the gestures made using the input device and the changes to be made upon the image. The one to one relationship makes it possible to paint directly onto any layer even if there are layers above it and below it.
One problem with the layer based paint system is that if the artist subsequently wishes to modify a layer, first the processing of the layer must be undone. For example, if the artist rotates a layer, the rotation is applied to the pixels in the image raster to provide an image layer with new, rotated pixel values. To blur the image, a blurring process is applied, updating the image raster with a new, blurred image. To remove the rotation, the process of blurring must first be undone before the rotation is reversed. Once the rotation is reversed, the processing is reapplied to the image. The iterative reprocessing steps performed when modifying an image layer are undesirably time consuming. In addition, repetitive reprocessing of the image may result in the addition of visual artifacts that detract from the quality of the final image.
More advanced layered paint and animation systems use object based architectures. Each layer is an object that may contain an image, an image process (e.g., a blur) that applies a function to the image, or brush strokes, which change the color of selected pixels within the image. Each object retains information about its position, scale, rotation, etc. Because image layers are objects that contain an original image, rotation and scaling information, there is no longer a simple relationship between the gestures a user might make with the input device and the corresponding changes that would have to be made to paint directly on the image raster. As a result, object based paint systems typically dictate that any new paint is applied to a new layer. Accordingly, in such systems it is not generally possible to directly modify the image raster associated with the object without creating a new, modified object. Rather, any modifications to the image raster are made in layers above the image raster.
Known object based systems do not typically allow the effects of processes to be viewed when adding brush stroke layers beneath the process layers. Thus, if a user wishes to modify an image layer, the painted layer is typically generated above the process layers, and when the painting is complete the layers are reordered and the processes may be viewed. Because processing is not done directly to the images, there is no degradation of the image. However, the additional layering steps are time consuming and inconvenient.